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ABSTRACT 



This report analyzes the 1999 Student Satisfaction Inventory 
(SSI) survey results of Salt Lake Community College (SLCC) (Utah) conducted 
by the Noel-Levitz company. In 1999, a stratified random sample was utilized 
that produced a more representative profile of the student demographics 
compared to previous years (e.g., the percentage of full-time to part-time 
students in the survey closely reflects the profile for the entire student 
body) . SLCC was rated highest on: responsiveness to diverse populations, 
registration effectiveness, and academic services. SLCC was rated lowest on: 
admissions and financial aid, academic advising, and campus support services. 
Discussions are provided on the following topics: the validation of scale 
results, program areas with the lowest SSI ratings, examination of the 
"neutral" response, further analysis of the lowest SSI ratings, and 
implications and recommendations for further research. The effectiveness of 
the registration procedures and satisfaction with Academic Services is 
independently supported by high ratings in the new student and graduating 
student surveys which are conducted every year. No data were available to 
independently validate the high rating on responsiveness to diverse 
populations. The difference in the scores may well be due to the difference 
in the questionnaire wording. (VWC) 
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Introduction 



The Noel-Levitz company provides a statistical report of the results of the 
Student Satisfaction Inventory (SSI) that concentrates on the mean overages of 
the student responses evaluating SLCC programs and services. This year the 
company was asked to send a copy of the electronic file of the data base in 
order to analyze the survey results in greater depth. This permitted the analysis 
that follow. 



The Student Sample 

One of the goals of the Noel-Levitz survey at Salt Lake Community College is to 
hove results that ore representative of the entire student body. 
Representativeness depends upon having a very good random sample. In 
1999, a stratified random sample was utilized that produced a more 
representative profile of the student demographics compared to previous years 
(e.g., the percentage of full-time to part-time students in the survey closely 
reflects the profile for the entire student body). 



Overview of the Survey Results 

The survey questionnaire contains roughly 100 questions which ore grouped into 
1 2 subject scales with on overage evaluation rating for each scale. The 
maximum range for the scale scores would be 1 .00 for "very dissatisfied" to 7.00 
for "very satisfied" . 

The range of scores for the 1 2 scales at SLCC in 1999 was a low of 4.64 to a high 
of 5.23 



SLCC was rated highest on: 

Responsiveness to Diverse Populations 5.23 

Registration Effectiveness 5. 1 2 

Academic Services 5.07 

SLCC was rated lowest on: 

Admissions and Financial Aid 4.64 

Academic Advising 4.67 

Campus Support Services 4.69 
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Validation of the Scale Results 



The effectiveness of the registration procedures and satisfaction with Academic 
Services is independentiy supported by high ratings in the new student and 
graduating student surveys which are conducted every year. No data was 
avaiiabie to independentiy vaiidate the high rating on responsiveness to diverse 
popuiations. 

The fourth ranked program in the survey is instructionai Effectiveness with a 5.05 
average rating. SLCC conducts comprehensive student ciass evaiuations every 
year. For the past five years the resuits have consistentiy shown an average 
ciass rating of very good (4.00 on a 5.00 scaie). These SLCC student ciass 
evaiuations provide the most conciusive data available because they are the 
most representative results for the entire student body. While " Instructional 
Effectiveness" in the SSI is one of the highest rated programs, it is interesting to 
note that the average score of 5.07 on a 7-point scale is comparatively lower 
than the average for all classes of 4.00 on a 5.00 scale. The Noel-Levitz scale 
score would have to be 5.60 to correspond to the official SLCC rating system. 

The difference in the scores may well be due to the difference in the 
questionnaire wording. The SLCC class evaluation system is straightforward and 
asks for qualitative ratings (e.g., good, very good, etc.) to a general subject 
(e.g., quality of class, effectiveness of the teacher, etc.). The highest rating for 
any of the questions in the Noel Levitz Instructional Effectiveness scale is: " The 
quality of instruction I receive in most of my classes is excellent." The student is 
then allowed to respond in terms of a range of choices from "very dissatisfied" 
to "very satisfied." The students are responding to a specific concept which 
contains the word " excellent." The percentage " satisfied" or " very satisfied" 
could well have been higher if the students were responding to the generic 
subject " the quality of instruction." These results illustrate why questionnaire 
wording is a critical subject in survey research. 



Program Areas with the Lowest SSI Ratings 
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The SLCC program areas ranked lowest on the Student satisfaction inventory 
were: 

"Admissions and Financial Aid" 4.64 "Academic Advising" 4.67, "Campus 
Support" 4.69. In the Noel Levitz scoring system, a score of 4.60 is just above the 
midpoint between 4.00 " neutral" and 5.00 "somewhat satisfied." 

Further examination of the "Campus Support" scale found a problem in the 
data that suggests no conclusions should be drown from these results alone. 

This scale focuses primarily on programs that ore used by very small 
percentages of the student body (e.g. one and three percent) The scores on 
four of the questions in this scale (e.g.. Veteran's Services) were significantly 
lowered because of the number of respondents who used the " neutral" 
response when they hod the option of the " not used" response. The " neutral" 
responses, with a score of 4.00, become a port of the overage score 
calculation. 

To what extent is this methodological situation a potential factor in other 
program areas? It appears to apply to a number of program areas. 

Examination of the “Neutrai” Response 

The Noel-Levitz questionnaire contains eight response options. The " not 
ovoilable/not used" response is not port of the evaluation score calculation. 
There ore three response levels of satisfaction and dissatisfaction and a 
" neutral" response option. All seven ore used in the scoring calculation. With a 
" not used" response option available, the " neutral" response is available for 
persons who did use and ore familiar with the subject in question, but who do 
not wont to take either a "satisfied" or dissatisfied" position. Perhaps their 
feelings ore too uncertain or mixed, or they simply don't hove on opinion. The 
" neutral" response receives a score of 4 on the 7-point scale. The "somewhat 
satisfied" response receives a score of 5. The overage rating scores for oil SLCC 
program areas and the individual questions that moke up a program area 
tend to range between the high 5's and low 4's. A large percentage of 
" neutral" responses on individual questions will bring a program area score 
down. The question is, in such coses, ore the lower scores a valid reflection of 
dissatisfaction and program problems? 

The following table provides examples of what could be argued ore normal 
amounts of uncertainty about a question os reflected in the percentage of 
"neutral" responses. 

SSI Questions with the Lowest Percentage of " Neutral" Responses 
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Questio 

n# 


Question 


Percentage of 
"Neutral" (#4) 
Responses 


8 


Classes are scheduled at convenient times 


8.7 


69 


A good variety of courses are provided 


9.9 


97 


Overall satisfaction with SLCC experience 


11 


39 


Amount of student parking space on campus is 
adequate 


13.7 



The potential problem can be seen in the next table. Roughly one percent of 
the student body has contact with the child care center and perhaps four 
percent with the personnel in the Veteran Services Office. In the question 
dealing with the latter program, 38.3 % responded they did not use the office. 
Seventy percent of those included in the rating score for this program marked 
the " neutral" response (43.4% of 61 .7%). The lower evaluation score on this 
question does not validly establish the conclusion that there is a problem in this 
program area. 

This observation also applies to the lower scores received by the child care and 
displaced homemaker programs. It may also apply to the security staff 
evaluations. 



SSI Questions with the Highest Percentage of " Neutral" Responses 



Questio 
n # 


Question 


Percentage of 
"Neutral" (#4) 
Responses 


4 


Security staff helpful 


41.5 


10 


Child care facilities are available on campus 


45.8 


11 


Security staff respond quickly in emergencies 


44.5 


17 


Personnel in Veterans' Services helpful 


43.4 


19 


Effective support services for displaced homemakers 


41.7 
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Large percentages of " neutral" responses may also affect the programs related 
to the following questions. 



SSI Questions with the Second Highest Percentage of " Neutral" Responses 



Questio 

n# 


Question 


Percentoge of 
"Neutrol" (#4) 
Responses 


9 


Internships are provided in my program 


35.0 


33 


Academic counselors accurately portray the 
campus in recruiting 


34.9 


30 


Career services provide the help students need 


33.5 


44 


1 generally know what's happening on campus 


33.4 


49 


Admissions counselors respond to student's unique 
needs and requests 


32.4 


13 


Financial aid awards ore announced in time to be 
helpful in college planning 


32.4 


20 


Financial aid counselors ore helpful 


30.4 


55 


Academic support services adequately meet 
student needs 


30.3 



Further Analysis of the Lowest SSI Ratings 

The lower rating for the admissions and financial aid programs deserves closer 
analysis. To do this, the " did not use" and the " neutral" responses were token 
out of the calculations for the question " Financial aid counselors are helpful. 

That eliminated 55 percent of the total respondents. The resulting calculation 
found that 71 were satisfied on this subject and 29 percent were dissatisfied. The 
first point of analysis is that the majority of respondents were positive in their 
evaluation. But the 29 percent negative is comparably higher than most other 
questions in the SSI survey. It is not possible to know why there were that many 
negative responses based solely on the questions in the SSI. Was it student 
expectations, a problem with the staff persons involved, the system process 
itself, or some other reason? Only further research and investigation could 
provide the answer. 
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Academic advising at SLCC is a iarge muitifaceted effort, it is conducted by 
staff and focuity in muitipie iocotions and programs. The SSi does not provide 
any information on the student's frame of reference. Do the ratings appiy to aii 
parts, or only certain segments of the advising program? Furthermore, only 
additionai research wouid clarify the extent that any probiems ore ottributobie 
to the staff, the system or some other factor. But the questions that comprise this 
scaie do provide some vaiuobie data and insights. 

The two reiated questions with the lowest scores are: " My academic advisor 
helps me set goais to work toward" and, " My academic advisor is concerned 
about my success as an individual." Anecdotai reports suggest that most 
students meet with an academic advisor only to schedule ciasses for the 
upcoming semester. Long range career and academic pionning is o different 
need and subject. The lower score on "setting goais" draws attention to the 
need for just such long range pionning with some students' 

What might be the extent of the probiem? Exciuding the " did not use" and 
" neutrai" responses the percentage of positive and negative evaiuations were 
caicuiated. 



Results from Three Questions in the Academic Advising Scaie 




Positive 


Negative 


My academic advisor helps me set goals to work 
toward. 


64% 


36% 


My academic advisor is concerned about my 
success os on individual 


67% 


33% 


My academic advisor is knowledgeable about my 
program requirements 


75% 


25% 



The possibiiity that between one-quorter and one-third of the students who use 
the academic advising program are dissatisfied strongiy invokes the need for 
further investigation and research into the program area. 

Finoliy, it is interesting to note that out of the whole questionnaire, the two singie 
questions that receive the lowest rating are not a part of the iowest ranked 
program areas examined above. Those are a score of 4.15 on the adequacy of 
parking and a score of 4.14 on the item " Students are notified eariy in the term 
if they are doing poorly in a cioss." Both of these subjects have received 
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similarly lower ratings on other surveys. The one on feedback about academic 
progress during the semester is a subject that should be examined further by the 
faculty. 

A Comparison of SLCC and Weber State University Student Ratings 

Many of the some questions were asked at SLCC and Weber State University in 
lost year's Noel Levitz Surveys. A selective comparison is presented below. 

SLCC's overall evaluations ore higher than those for Weber State. But the 
advising and financial aid programs again hove lower comparable ratings. 
Additional Information is necessary before it con be known exactly what the 
problem Is, where It Is o problem and why it is o problem. 



Comparable Student Evaluations on the General Campus Environment 


Question 


SLCC 


Weber State 


It is an enjoyable experience to be a student on this 
campus 


5.26 


5.04 


Students ore mode to feel welcome on this campus 


5.24 


4.95 


This institution has a good reputation within the 
community 


5.37 


5.31 



Comparable Student Evaluations on the Academic Advising Programs 


Question 


SLCC 


Weber State 


My academic advisor is approachable 


4.92 


5.02 


My academic advisor helps me to set goals to work 
toward 


4.34 


4.55 


My academic advisor is concerned about my 
success os on individual 


4.44 


4.85 


My academic advisor is knowledgeable about 
requirements in my major. 


4.81 


5.19 
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Comporobie Student Evaiuotions on the Finoncioi Aid Programs 


Question 


SLCC 


Weber State 


Financial aid counselors are helpful 


4.48 


4.63 


Financial aid awards are announced to students in 
time to be helpful in college planning. 


4.27 


4.67 


Adequate financial aid is available for most students 


4.66 


4.38 


My academic advisor is knowledgeable about 
requirements in my major. 


4.81 


5.19 



Recommendations 

Continue to use the Noel-Levitz survey with one important modification in the 
survey procedure, instructions to the students participating in the survey shouid 
expioin the purpose of the " neutroi" versus the " did not use" response options. 
Drawing this to their attention shouid iower the number of " neutroi" responses. 
All other things being equoi, the resuit shouid be a raising of the overage rating 
scores. 

it is also important to properly impress on the students the importance of the 
survey and their participation, it is better that they do not respond to any one 
question than to arbitroriiy mark one response (e.g. #4) to a iong series of 
questions. 

Continue to order the survey data disk from Noel-Levitz so that the percentage 
of positive and negative responses con be coicuioted for individuoi questions. 

Recognize that the Noei-Levitz survey occompiishes its purpose by providing a 
brood overview of student satisfaction. But brood overviews, by nature, ore 
reiotiveiy superficioi. When results suggest that a problem may exist, the 
appropriate next step is to investigate the situation in greater depth. A variety 
of methodoiogies couid be used to gather further information. This inciudes: a 
foiiow-up survey with questions that focus on specific aspects of the situation 
(e.g. where is the probiem, what is the specific probiem and why is it a 
probiem). A questionnaire survey, or focus group discussion, with specific 
students or staff and focuity ore vaiuobie research options. Good research 




10 



9 



typically raises further questions. Research, evaiuation and program 
improvement need to be understood as an on-going evoiutionary process. 
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